1. Technical Field
The present disclosure relates generally to human-computer interfaces and mobile devices, and more particularly, to motion-based graphical input systems.
2. Related Art
Mobile devices fulfill a variety of roles, from voice communications and text-based communications such as Short Message Service (SMS) and e-mail, to calendaring, task lists, and contact management, as well as typical Internet based functions such as web browsing, social networking, online shopping, and online banking. With the integration of additional hardware components, mobile devices can also be used for photography or taking snapshots, navigation with mapping and Global Positioning System (GPS), cashless payments with NFC (Near Field Communications) point-of-sale terminals, and so forth. Such devices have seen widespread adoption in part due to the convenient accessibility of these functions and more from a single portable device that can always be within the user's reach.
Although mobile devices can take on different form factors with varying dimensions, there are several commonalities between devices that share this designation. These include a general purpose data processor that executes pre-programmed instructions, along with wireless communication modules by which data is transmitted and received. The processor further cooperates with multiple input/output devices, including combination touch input display screens, audio components such as speakers, microphones, and related integrated circuits, GPS modules, and physical buttons/input modalities. More recent devices also include accelerometers and compasses that can sense motion and direction. For portability purposes, all of these components are powered by an on-board battery. In order to accommodate the low power consumption requirements, ARM architecture processors have been favored for mobile devices. Several distance and speed-dependent communication protocols may be implemented, including longer range cellular network modalities such as GSM (Global System for Mobile communications), CDMA, and so forth, high speed local area networking modalities such as WiFi, and close range device-to-device data communication modalities such as Bluetooth.
Management of these hardware components is performed by a mobile operating system, also referenced in the art as a mobile platform. The mobile operating system provides several fundamental software modules and a common input/output interface that can be used by third party applications via application programming interfaces.
User interaction with the mobile device, including the invoking of the functionality of these applications and the presentation of the results therefrom, is, for the most part, restricted to the graphical touch user interface. That is, the extent of any user interaction is limited to what can be displayed on the screen, and the inputs that can be provided to the touch interface are similarly limited to what can be detected by the touch input panel. Touch interfaces in which users tap, slide, flick, pinch regions of the sensor panel overlaying the displayed graphical elements with one or more fingers, particularly when coupled with corresponding animated display reactions responsive to such actions, may be more intuitive than conventional keyboard and mouse input modalities associated with personal computer systems. Thus, minimal training and instruction is required for the user to operate these devices.
However, mobile devices must have a small footprint for portability reasons. Depending on the manufacturer's specific configuration, the screen may be three to five inches diagonally. One of the inherent usability limitations associated with mobile devices is the reduced screen size; despite improvements in resolution allowing for smaller objects to be rendered clearly, buttons and other functional elements of the interface nevertheless occupy a large area of the screen. Accordingly, notwithstanding the enhanced interactivity possible with multi-touch input gestures, the small display area remains a significant restriction of the mobile device user interface. This limitation is particularly acute in graphic arts applications, where the canvas is effectively restricted to the size of the screen. Although the logical canvas can be extended as much as needed, zooming in and out while attempting to input graphics is cumbersome, even with the larger tablet form factors.
Expanding beyond the confines of the touch interface, some app developers have utilized the integrated accelerometer as an input modality. Some applications such as games are suited for motion-based controls, and typically utilize roll, pitch, and yaw rotations applied to the mobile device as inputs that control a on-screen element. Along these lines, more recent remote controllers for video game console systems also have incorporated accelerometers such that motion imparted to the controller is translated to a corresponding virtual action displayed on-screen. Accelerometer data can also be utilized in other contexts, particularly those that are incorporated into wearable devices. However, in these applications, the data is typically analyzed over a wide time period and limited to making general assessments of the physical activity of a user.
Because motion is one of the most native forms of interaction between human beings and tangible objects, it would be desirable to utilize such inputs to the mobile device for controlling user interface elements thereof. It would also be desirable to expand the canvas in graphical design applications, and to capture motion imparted to the mobile device as input strokes that are translated to graphics on the screen.